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CLOZE IN THE CLASSROOM WITH NOTES 
ON NEEDED RESEARCH* 

Virginia Streiff 

Interest in cloze as an effective and economical language 
measure has been increasingly visible in scholarly research. 
• Studies abound which report the use of cloze as a valid and reli- 
able language proficiency measure, from initial studies (Oiler and 
Conrad, 1971; Oiler, 1 972; Oiler et al. , 1973) to more recent 
ones (Stump, 1978; Streiff, 1978, 1979a, 1979b; Hanzeli, 1977; 
Hinofotis, 1980). Questions on the effectiveness of certain as- 
pects of cloze have also been raised (Alderson, 1979? Porter, 
1978). Oiler (1979) recommends cloze as a valuable measure for 
classroom language testing. However, only sparse information ex- 
ists about how cloze works for the classroom rather than in re- 
search projects. A series of classroom cloze testing projects 
conducted by ESL and bilingual teachers has yielded valuable prac- 
tical information for classroom teachers. These projects also 
suggest additional research topics. 

The basic question addressed in this report i.u What hap- 
pens when the cloze procedure is utilized by classroom teachers 
who are not extensively schooled in testing in general or in lan- 
guage testirg in particular? The focal points for discussion 
are: 



*This paper was presented at the 14th Annual TESOL Convention, San 
Francisco, California, March, 1980. 
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1. material selection for clofce; 

2. some technical aspects of construction and administra- 
tion? 

3. insights about testing language students naive to cloze; 

4. reliability of cloze administered to small groups; and 

5. typical native speaker scoring patterns, which may sug- 
gest criterion ranges for non-native speaker cloze score 
comparison. 

The findings are drawn primarily from the work of graduate 
students who participated in a language testing seminar in 1979. 
The seminar was a cooperative enterprise in which students met for 
a series of classes, and then pursued projects with their own stu- 
dents. The projects were implemented in classrooms from kinder- 
garten through adult levels among monolingual English speakers, 
Spanish/English bilinguals, and adult ESL learners. Because we 
can learn from mistakes, the mistakes as well as the successes 
will be mentioned. Some insights thus gained obviously lead back 
to the tenets of standard tests and measurements texts. However, 
others are specific to use of the cloze procedure. The main 
findings and recommendations are offered below. 

MATERIAL SP r »ECTION 

Basically, two types of materials were used. Most were pas- 
sages drawn from published texts and library books. However, in 
several instances, teachers followed the suggestion of Streiff 
(1978) and collected oral child narrative to transform into oral 
or written cloze passages. Figure 1 presents the main recommenda- 
tions forthcoming from the teachers' studies. Both the materials 
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Figure 1 



MATERIAL SELECTION FOR CLOZE 



Grades K through 6 

Avoid: oral interview material that is lengthy and detailed on a 
single topic for general language proficiency purposes, 

published material that is very difficult in readability 
or very unfamiliar in topic for students, and 

stories that are highly idiosyncratic to environments or 
cultures very different from those of students, especially 
very unusual i.ames of characters or places. 



High School and Adult 

Avoid: very juvenile story topics that may be "easy" but an 
insult to adult learners, 

material that is very difficult in readability level, even 
though students are older. 



General Points 

1. While conducting an oral interview to be used for cloze mate- 
rial, keep topics moving along with questions and statements. 

2. Use published material w^th natural breaking points at begin- 
ning and end of test material. 

3. Use material that will be equally familiar or unfamiliar to 
all students. 

4. Use instructional material from your own classroom. 
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* that worked well and the problems to avoid are presented below in 
grade level sequence. 

Kindergarten and First Grade 

Child narrative refers to child discourse obtained during an 
oral interview, which is tape recorded, transcribed, then edited 
lightly by adding a short introductory section and transition sen- 
tences between topics. Words are then deleted from the child nar- 
rative portions of the final passage. Interview topics that have 
worked well for cloze material at these grade levels include: 
family members? the child's favorite activities, foods, and 
friends anecdotes about family pets; current and typical school 
activities? and the child's observations about teachers. Cloze 
passages constructed from such material have been found to be both 
comprehensible and interesting to children from Native American, 
Spanish/English bilingual, and monolingual backgrounds. 

Specific themes from published material that have worked well 
include a reading series story abut a child who doesn't smile un- 
til "show and tell" time at school because she wants to "show" 
that she has lost a tooth ? a story of a little child who brings 
each of his favorite objects one at a time to sit with him on his 
mother's lap? and a fable about an old man, an old woman, and 
their animal friends who all help pull a giant turnip out of the 
ground. Highly repetitious tales were particularly effective with 
kindergarten children. 

Two problems were encountered at this level with material se- 
lection, one teacher interviewed a kindergarten child for mate- 
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rial and found herself listening to a lengthy and detailed account 
of a movie the child had seen. She tried her cloze test made of 
this material with a few other children and found that they 
quickly grew disinterested in another child's long account of the 
movie. The teacher then reduced this content and incorporated 
other topics the boy had mentioned and found much more satisfac- 
tory response when she administered the final test. A second 
problem to avoid is constructing cloze from story material too 
difficult for the grade level being tested. One teacher construc- 
ted two tests, one at first grade readability level and one at the 
third grade level. S». administered the tests in a first grade 
other than her own. That classroom's teacher was interested in 
knowing what the children could do, since she herself was new to 
this particular classroom. The third grade story was exceedingly 
difficult for most of the first graders. The seminar student was 
understandably concerned about the task discouraging the children. 
However, one good outcome was that two of the children actually 
did so well on the harder cloze that their own teacher immediately 
provided them more appropriate material for reading instruction. 
An alert teacher could administer a series of increasingly more 
difficult passages to students, allowing them to stop when the 
work becomes too difficult and determine instructional levels 
without unduly difficult demands upon the young readers. 

Grades Two through Six 

Child narrative, as described above, may be used success- 
fully. Additional topics include special skills and interests a 
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child has developed and special accomplishments such as getting a 
job or participating in sports events. Especially effective mate- 
rials were natural science topics and other factual articles from 
junior level encyclopedias. Topics that posed problems and that 
should be avoided were stories highly idiosyncratic to environ- 
ments or cultures very foreign to the students' own and stories 
containing unusual names. American children, regardless of cul- 
tural background, appear to have very similar reading topic pref- 
erences, as Streiff (1979b) found in reviewing reading interests 
of Eskimo and mainstream American children. 

High School and Adult Levels 

Appealing oral interview material for cloze tests directed at 
foreign students included the followng: other foreign students' 
first impressions of this country, problems encountered, and 
pleasant surprises about the United States j school and leisure ac- 
tivities, and differences between this country and the students* 
home countries. A pertinent suggestion for teachers who teach 
content areas other than language to foreign students is to tape 
record their own teaching presentations and use portions of these 
recordings as language test material. Such cloze content would 
reveal quickly how comprehensible the teacher is to the student. 
Also, portions of television newscasts can be recorded and tran- 
scribed for use as cloze tests. 

Reader's Digest articles, passages from ESL texts, and 
foreign language texts have also worked well for older students. 
Of particular value are articles from elementary and junior high 
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level encyclopedias. Seminar students found the topics of rivers, 
§ electricity, and astronomy from these sources quite effective, 

especially for older students who were educated in their home lan- 
guages but who needed easier reading material in English. Neutral 

- topics with about the same degree of familiarity or unf amiliarity 
for all students were chosen. One teacher, looking for easy mate- 
rial, initially selected a child's book about a froggy woggy and a 

- ducky wucky in a puddle wuddle! He finally selected a more so- 
phisticated topic but at an easy readability level. For these 
more mature ESL students, the most satisfactory sources of mate- 
rial seemed to be elementary and junior level encyclopedias. The 
idea of using children's literature should not be discarded, how- 
ever. For parents who are ESL students, good children's litera- 
ture could be of great interest and also of practical value for 
both parent language learners and their children. 

General Points about Material Selection 

For oral cloze material, the teacher-interviewer should keep 
the material -gathering interview moving along with questions, en- 
couraging responses, just as a good human interest writer would do 
for a news report. If one topic becomes too detailed or idiosyn- 
cratic, the material can be edited out later as the test is pre- 
pared. An alternative to interviews is to tape record conversa- 
tions among native speakers of the language to be tested. Hughes 
(1980) reported this technique to yield a successful written cloze 
test among his subjects. With increasing definition of the impor- 

- tance of speech events and speech acts to language acquisition and 
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use (Hatch, 1978? Fraser, 1979? Canale and Swain, 1979), addi- 
tional potential for interview and conversation-based cloze tests 
is suggested 5 for example, conversations where apologies are of- 
fered, different types of requests are made, and where greetings 
and leavetakings are occurring in different contexts. 

The most effective published material for cloze content had 
passages with topically natural breaking points at beginning and 
end. Topics selected should be of equal familiarity, or as close 
as possible, for the test-takers. 

Cloze is adaptable to most kinds of normal classroom instruc- 
tional materials, from library books to training manuals in tech- 
nical subjects. A teacher can discover objectively and quickly 
what materials students can handle and how much they progress. 
Thus, an obvious source for cloze is on the classroom shelves. 
However, some teachers may have problems here, as some foreign 
language teachers discovered when they wanted to develop pragmatic 
tests for their students? Thei" texts had no intact passages in 
them, containing only explanations of grammatical rules, short 
pattern drills, and vocabulary lists. 

Examples of Cloze Passages Used by Seminar Students and Others 

Following are some samples taken from cloze tests used at 
various age/grade levels. 

Kindergarten, Oral Cloze (Sabino, 1979) 

Once upon a time there lived an old man and an old woman who 
planted a turnip. The turnip grew and grew. At last it was 

ready to pull. The old man tugged at the turnip. 

1 
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pulled and he tugged.// He tugged and .// But the 

turnip would not come of the ground.// Then the 

5 

old man the old woman.// The old man tugged 

4 

the turnip.// The old woman tugged the 
5 § 
old man. 



Kindergarten, Oral Cloze (Ruvalcaba, 1979) ' 

Then David talks about his family and what they like. "My 

mother's name is Olivia. father's name is 

5 

Raymond.// The inside of my house white.// My mom 

7 

likes owls? boy, she really them.// My mom and dad 

§ 

and myself go to movies.// Then we all go shopping 

in the mall." 



Grade 1 , Oral C!o2e (Correa, 1979) 

This is a story about David. He is almost seven. David 
likes to talk about animals. In this story, David will tell 
us about a classroom project on dinosaurs. **I like dino- 
saurs. Today we're going to work on our dinosaur centers 
here at school. There are seven children in each center, and 
so we have four dinosaur groups. The teacher will give each 
one of us some clay. We .will make dinosaurs out of clay. 



( In ) my group I will work with (my) best friend 
1 ? 

Ernest. Ernest and I (are) very good friends. He is very 

3 

( good ) at making dinosaurs out of clay. ( I ) am very 
4 5 



good at making turtles (out ). of clay. Our group plans to 

6 

(draw) rivers and volcanoes on construction paper." 
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Grade 4, Oral Cloze (Streiff, 1978) 

Thomas is just nine years old, but he has a job. In this 

part of the story, Thomas told about why he got a job, how he 

got it, and what he does. Thomas decided to look for a job 

one Saturday when everyone in his family was busy but him. 

Thomas said, "I just told my mom that I was real / / 

IB 

an 1 I just told her I want to go a job."// She 

T5 

said, "Go ahead." Thomas said, "She just thought, 'Who's 

going hire you?'"// Thomas rode his bike down to 

3~Q 

the store. He said, "I asked one of the workers. Then 
said I could have a job."// Thomas said, "I came 



31 

back I said, 'Mom, I got a job.'// And she 

32 33 

'What?' And I go, 1 1 got a job!'"// And cjo^s, "You 

34 

got a job?"// Then Thomas said, "You said I could 

35 

a job." 

Grades 3-4, Written Cloze (Ferns and Ball, 1979) 

Few people ever see a beaver in the wild. And that is what 

the beaver wants. Millions of beavers have been killed by 

• So the beaver seeks a lonely or pond, 

1 2 

away from towns and . Re generally works and eats 

3 

at beca* se he does not like to himself in 

? 5 

the daylight. 

Grade 6, Oral Cloze (Streiff, 1978) 

Then Glenn said, " of my friends that are on the 

31 

track team to my school.// Sometimes I go over to 

33 
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t*- ir and then we skateboard.// We have a lot of 

,// and sometimes we ride doubles on the skateboard. 

We to do tricks."// 

18 



High School, Written Cloze (Brooke and Chernoff, 1979) 
It is very late at night in the city of Mexico. The streets 
are empty. It is quiet, quiet. A man walks alone along a 
dark *, his footsteps echo against the tall buildings 

—m — 

like pistol shots. The silence, and the emptiness 

(J) 

of the streets, makes the man feel . Suddenly the 

(3) 

quiet is shattered by a terrible shriek, followed by 



(4) 

long mournful wail. What is it? The man doesn't know which 
to turn. The sounds seem to come from all around. 



(5) 

Adult, Written Cloze (Autry, 1979) 

Electricity is a form of energy. Ke know that it is by what 

it does. Electricity (gives ) us light, heat, and power. 

It brings us ra£io f television, (motion) pictures, and the 

2 

telephone. 

Electricity comes into our (homes ) and schoolrooms on 

5 

wires. The electric current follows a (wire) circuit, usu- 

5 

ally made of copper or aluminum wires. Electric (current) 

5 

is easily turned on or off by switches. 

CLOZE CONSTRUCTION AND ADMINISTRATION 

Construction 

For knowledge of the independent, instructional, and frustra- 
tional reading levels of students, the work of Bormuth (1968) and 
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Peterson et al. (1973) establishes the requirement for dele- 
tion of every fifth word for readers from elementary school 
through college and adult vocational training levels. The latter 
work re-validated the 57 to 60 percent (exact word score) range as 
reflecting independent reading capacity. 

However, for sampling the general language proficiency (ver- 
sus reading levels) of learners, the seminar students universally 
preferred the more lenient schemes of every seventh or eighth 
word. Haskell's (1973) findings support this preference. The 
seventh word deletion scheme seems to provide the best balance be- 
tween test effectiveness on the one hand and minimizing test 
length and test-taker frustration on the other. 

Seminar students recommended using easier rather than more 
difficult material, particularly for audiences new to cloze 
testing, for the initial cloze experience. They expressed concern 
over clarity of test copies; one teacher was dissatisfied with the 
blurry results produced by the school ditto machine. The result 
actually amounts to "visual noise" in a cloze test, a topic of in- 
terest in language test research but not appreciated by the class- 
room teacher to whom student reactions are important- Most teach- 
ers of young children allowed for their students' large print by 
allowing extra long lines for blanks and extra space between 
lines. One who didn't thought it would have been a good idea, 
since the children's writing overlapped printed words above and to 
the side of the blanks. 

After proofreading their cloze tests, several teachers made 
these recommendations: (1) leave several sentences intact at the 
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beginning of the test, and (2) be sure the number of word's between 
blanks is the same ^throughout the test. The effect of moving a 
blank to the right or left is negligible (Bormuth, 1964); however, 
it then becomes tempting to deliberately leave in certain words 
that the pre-selected deletion scheme would remove. While further 
cloze studies for researchers may be indicated, it seems wisest 
for classroom cloze users to follow the established procedures. 

Administration 

Two interesting ideas that were suggested for kindergarten 
teachers were use of a flip chart and of a noise Jignal for blanks 
during oral cloze testing. The flip chart provided a large visual 
reproduction of the story with blanks in place for visual refer- 
ence. Two teachers tried this technique to provide students with 
added cues during the test. One, working with monolingual English 
speakers in a private kindergarten, found t; at the flip chart ad- 
ded nothing to test effectiveness. Another observed that it 
seemed to help less proficient students but was not needed for her 
more capable students. This technique appears worthy of further 
study. The same teacher used a Halloween noiseraaker with a click- 
ing sound to signal blanks during the oral cloze, rather than sim- 
ply saying the word "blank. " Again, she noted that it seemed to 
help the less capable students but not others. 

Two of the major points forthcoming from primary and elemen- 
tary grade projects are interrelated! administering cloze in 
shorter segments and providing breaks during testing sessions. 
The suggestion to divide the 50-item cloze tests into two shorter 
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tests of 25 blanks each was made after several projects. The same 
Z teachers concluded that they should have anticipated building in 

breaks for their students part way through the test. In two 
cases , teachers kept the students working for unduly long periods 
£ of from 60 to 90 minutes on written cloze tasks. This demand 

seemed to arise out of teachers' efforts to have children try very 
hard to supply vords even when it was obviously too difficult for 
_ them. The teachers were evidently r convinced in advance that 

the "skip it" rule built into previous oral cloze testing with 
young children (Streiff, 197C) had misrit. This view may arise 
ftom teachers who are accustomed to tests on which the "best" stu- 
dents achieve scares of 100 percent or close to it. This is not 
typical for cloze performance, particularly among kindergarten 
children who typically fail to supply function words, hor oral 
cloze, it is strongly recommended that teachers read cloze mate- 
rial alrnc as a good story, not as a series of isolated test sen- 
tences. R peating ov< r and over with a single sentence while 
children puzzle only makes titeru looe track of the important over- 
all meaning of the pasage, which is what helps them fill in the 
words in the first place. Listener motivation declines and the 
pragmatic value of the test i3 lost. 

Though the points may seem obvious, here are two more recom- 
mendations from teachers' work witn individual oral cloze: (1) be 
sure the student has been to the bathroom in advance? and (2) 
don't administer the test during recess, special events, or chil- 
dren's favorite activities. 
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Teachers of high school and adult students found that test 
administration for an initial cloze experience proceeded most 
smoothly when students were assured they would receive the results 
of their tests and when students and teacher enjoyed good rapport 
and mutual respect. Length of test time was not a problem at 

these levels. In several instances, teachers administered cloze 
tests more than once. They reported that the second occasion pro- 
ceeded mvch more smoothly than the first, after students had re- 
ceived and reviewed initial cloze test results. This finding 
leads to the next topic, working with students naive to cloze pro- 
cedure. 

TESTING STUDENTS NAIVE TO CLOZE 

Several seminar students working with adult ESL students had 
difficulty eliciting interest in a cloze test. These students 
seemed more focused on passing one of several well-known multiple- 
choice ESL tests, which determine whether they can proceed to 
higher levels of education or training. In these cases, having 
good rapport with their students helped them get through the first 
cloze session. The adult ESL students seemed more relaxed and in- 
terested in trying the task on the next occasion. In one class, 
an adult ESL student whose instruction had largely been pattern 
practice, rule explanation, and vocabulary work commented, "This 
is really good. We should be doing this at least two or three 
times a week." Valette (1977) notes that initial negative reac- 
tion to cloze does not seem to be universal among older students. 
-Rather, it seems to be present among students whose main task is 
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to pass multiple-choice ESXi examinations used to determine entry 

Z and exit status of students. Two examples illustrate this point. 

Foreign language teachers who initiated cloze in their university 
classes were surprised at how interested the students were and how 

> hard they worked at it (Lantolf and Streiff, 1980). In one adult 

ESL situation, 72 students scored significantly better on a 
multiple-choice form of cloze than on a standard open-ended cloze 
with passages taken from the same chapter of a non-fiction work 
(X » 34.88 and 11.79, respectively? t - 20.97, £ - .000) (Oiler 
and Streiff, in press). It has been observed that multiple-choice 
cloze is easier (Hinofotis and Snow, 1980); but in this case, the 
test-givers observed that the students were also motivated. The 
students even commented that "circling the letters" wsn better 

I practice for the tests they would eventually have to take. 

Seminar students also found more and less effective ways to 
handle discussing cloze results with their students. An almost 

j chaotic response occurred when one teacher placed his correctly 

filled-in cloze model on the overhead projector and returned stu- 
dents' corrected tests to them. Students were understandably cur- 
ious about their own particular errors, and the teacher concluded 
that a better approach was needed. The next time, he again used a 
correct model on the overhead projector but returned a clean copy 
of the cloze test to each student. This time discussion and ques- 
tions proceeded in a more systematic manner. Students were then 
given their corrected tests to compare with the model, and the 
session was considered very profitable by both teacher and stu- 
dents. Following are suggestions from seminar students who admin- 
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istered cloze to older students who had never before experienced 
1 the procedure. 

1 . Use short cloze passages as teaching tools during regular in- 
struction for a period of several weeks or more before admin- 
istering a cloze test. 

_ 2. Administer a short preliminary cloze test and then give stu- 

dents a complete copy of the passage to study. 

3. Correct student work on cloze by using an overhead projector 
with responses in place* immediately after student independent 
work. 

4. Work with cloze regularly as an instructional technique. 

Interest in using cloze as a teaching procedure arose from 
this work with older students as well as from projects involving 
younger students. Figure 2 summarizes a sequence of steps extra- 
polated and refined from a cloze teaching procedure reported by 
Kingston and Weaver (1970). Classroom teachers of young second- 
language learners have reported that children enjoy the cloze 
learning tasks. Kingston and Weaver employed similar steps using 
language experience stories of rural Appalachian primary children 
and found significant benefits to their reading skills. 

RELIABILITIES OF CLOZE AMONG SMALL GROUPS 

The classroom groups involved in seminar projects mainly 
ranged from 5 to 25 students. Several larger groups are also in- 
cluded in this report. Test reliability is a key feature in eval- 
uating language measures. Two mechanical features that contribute 
to test reliability are larger, as opposed to smaller, numbers of 
test items and of test takers, while .90 is the reliability cri- 
j terion generally considered acceptable for standardized tests, 
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Figure 2 



A BASIC CLOZE INSTRUCTIONAL TECHNIQUE 



1 . Write short cloze passages on the board or on an overhead 
transparency. 

2. Supply a list of the appropriate words at one side of the 
board or transparency. 

3. Let students volunteer to read the passage aloud, selecting 
the words they think appropriate. Write them in as students 
give them. 

4. Provide students a written copy of the same passage and have 
them fill in their words independently. 



Variations 

1. Don't put any potential word fill-ins on the board or 
transparency. Rather, allow students to contribute as many 
words as they can for particular blanks. 

2. Use students' own discourse as the source of short cloze 
passages. 
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Valette (1977) suggests that a good classroom test will have a re- 
liability co-efficient in the range of .60 to .80. This lower 
range is accounted for partly by the small numbers of test takers 
usually involved, compared to those upon which standardized tests 
are normed. Table 1 presents KR-21 reliabilities of the cloze 
tests employed in the seminar students* projects. These figures 
are listed along with other pertinent information about the tests: 
the language of the test, the mode of administration (oral, writ- 
ten), and the scoring procedure (exact, acceptable). 

The reliability co-efficients fall within the desirable range 
for classroom tests and in 64 percent of the cases exceed .80. In 
the majority of cases, the acceptable word scoring procedure 
yields the more reliable scores, as Streiff (1979a) and Hinofotis 
(1980) report. The cloze tests clearly revealed the differences 
among abilities of students who comprised the test-taking groups 
at all levels from kindergarten through adult levels. Many of the 
seminar students observed that the cloze scores accorded well with 
their students' typical classroom work and grade averages. A sum- 
mary of the cloze reliabilities by major grade level groups is 
presented in Table 2. 

Some seminar students were able co obtain other kinds of test 
scores, along with the cloze scores, to analyze for criterion- 
related validity. Table 3 presents a sample of these correlation 
findings representing the range of grade levels. In a number of 
cases, the exact word scores are more substantial correlates of 
standardized test performance, as Bormuth (1968) found in exam- 
ining cloze scoring methods and standardized reading tasks. When 
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Table 1 

CLASSROOM CLOZE RELIABILITIES 



Teacher/Tester 


n of 


Test 


Admin. 


n of 


Score 


KRr-21 




Items 


Lang. 


Mode 


Subj. 


Type 


Reliabilities 


Kindergarten 
















Ruvalcaba* 


48 


Eng. 


oral 


20 




.71 


.82 






Span. 










Sabino 


34 


Eng. 


oral 


12 


accept. 


.77 


.66 












exact 






Grades 1-2 
















Correa, 1 


50 






in 


exacr 


.92 
















.76 






SO 


Eng. 


writ. 


8 


exact 










Eng. 








.90 




Sanchez r 1 


50 


Eng. 


writ. 


20 


accept. 


.89 


.91 












exact 






Escalante* 7 




tug. 


oral* 


2 1 


accept. 


.97 


.99 








WL XL* 




exact 














accept. 


.99 


95 












exact 










Span. 


oral 




accept. 


.91 


c90 












exact 






Grades 3-5 
















Ferns and Ball 
















3 


50 


Eng. 


writ. 


16 


accept . 


.90 


.87 












exact 
















mult. ch. 


.88 




4 








23 


accept. 


.87 


.76 












exact 
















mult. ch. 


.83 




Taylor, 3-5 


50 


Eng. 


writ. 


6 


accept. 


.86 


.73 




47 








exact 








Span. 






accept. 


.96 


.91 



exact 
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Teacher/Tester 


41 SJL. 

Items 


•lest 
Icing. 


Aconin. 


n of 


Score 
Type 


KR-21 
reliabilities 


Streiff and 














Ibarra* 






« 








4 


50 


Etag. 


writ. 


23 


accept. 


.91 .82 












exact 








Span. 






accept. 


.90 .77 












exact 




5 








26 


accept. 


.92 .87 












exact 




* 










accept. 


.90 .85 












exact 




rxign ocnoox 














Anger t, 9-11 


50 


Eng. 


writ. 


85 


accept. 


.70 .65 


10-12 










exact 










45 


accept. 


.74 .63 












exact 




Brooke and 


50 


Ehg. 


writ. 


49 




77 

mil 


Chemoff 












Daniel 


50 


Germ. 


writ. 


16 


accept. 


.80 .62 












exact 




iV2ult Classes 














Autry 




Frvr 


writ. 


Q 

o 


accept. 


.70 


Powell 


49 


Ehg. 


writ. 


5 


accept. 


.70 




50 








.75 


Somrner 


50 


Eng. 


writ. 


16 


accept. 


.98 .96 




50 








exact 












accept. 


.99 .98 












exact 




Krueger, Berk, 


50 


Eng. 


writ. 


72 


exact 


.81 


DiQuinzio, 


50 








mult. ch. 


.88 



Ball, Osborn* 



* Random selection of test subjects. 
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Table 2 



SUMMARY OF CLASSROOM CwJZE RELIABILITIES 



Grades K-5 

N of subjects range from 6 to 26 
Acceptable word scoring 

Exact word scoring 
High School 

N of subjects range from 16 to 85 
Acceptable word scoring 
Exact word scoring 

Adult Classes 

N of subjects range from 5 to 72 
Acceptable word scoring 
Exact word scoring 



Range of KR-21 Reliab il ities 

.71-.96 
.66-. 99 



.7Q-.8Q 
.62-. 65 



.70-.99 
•81-.98 
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Table 3 

SAMPLE CORRELATIONS OP CLOZE WITH OTHER MEASURES 
Kindergarten (Sabino) 



Cloze 
Exact 
Accept. 
N - 12 



Tests of Basic Experiences 
Language Concepts 
-73 (.01) .69 (.01) 

-48 (.05) .66 (.01) 



Beabody Picture 
vocaf ilary Test 



.52 (.01) 
.48 (.05) 



Second Grade (Escalante) 




California Achievement Test 




Cloze 


Total Read. 


Total Lang. 


Total Batt. 


Oral Eng. Exact 


.79 (.001) 


.71 (.002) 


.68 (.001) 


Writ. Eng. Accept. 
N = 21 


.71 (.001) 


.72 (.001) 


.76 (.001) 



Fourth, Fifth Grades (Ibarra) 



4th 
N = 23 



5th 

N = 26 



Cloze 
Writ. Eng. Exact 
Writ. Eng. Accept. 
Writ. Eng. Exact 
Writ. Eng. Accept, 



Metropolitan Achievement Test 
Ward Knowl. Read. Con?). Read. Total 
.61 (.01) .72 (.001) .70 (.001) 

.64 (.001) .72 (.001) .71 (.001) 
.69 (.001) .58 (.01) .69 (.001) 
.62 (.001) .58 (.01) .66 (. 001 ) 



Adult (Krueger et al. } 

Correlations of Cloze and English Conprehension Levels with Single Factor 
N = 72 A. Factor B. Factor 

Standard Cloze .85 Mult. Choice Cloze .82 

.82 



English Comprehension 
Levels 



English Conprehension .83 
Levels 
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this finding is contrasted with the results of Hinofotis (1980), 
in which acceptable word scores yielded higher correlations than 
exact word scores with SSL tests, several points are suggested. 

First, there are some obv us differences between the present 
test situations and those of Hinufotis (1980), i. e., numbers of 
students in the groups, a«,.-a, and educational levels of the stu- 
dents, and particular types of tests (basically, elementary level 
achievement tests versus adult ESL tests). These different poten- 
tial sources of variance suggest the next point. Several sources 
of cloze performance variance have been investigated recently, 
such as passage-wide constraints versus short-range constraints 
(Chihara et al., 1977), alternate placement of blanks in equi- 
valent cloze forms (Porter, 1978? Streiff and Anger t , in press), 
and passage readability and distance between blanks (Alderson, 
1979). However, additional and more comprehensive categories are 
indicated by the empirically-based model developed by Klare 
(1976), derived from 36 recent research studies on readability. 
Klare concentrated on 28 reported variables toward the goal of re- 
assessing the validity of readability formulas. His findings are 
that reader performance is a function of five major variables in- 
teracting with one another: the test situation, reader compe- 
tence, readability level of the material, content of the material, 
and reader motivation. Such a model has value for cloze language 
proficiency studies, indicating predictable sources of variance 
and additional research topics as well. For example, there are 
ten variables in this model subsumed under "test situation." 
These variables comprise the sub-categories of; (1) motivating 



26 



factors, (2) test readability in relation to text readability, and 
(3) test effectiveness factors (e. g. , liberal reading time). The 
point, with respect to exact and acceptable scoring procedures and 
the varying correlations they produce with criterion measures, is 
that there are numerous systematic and interacting sources of var- 
iance. The remarkable feature of cloze is that under such varying 
conditions, cloze consistently yields acceptable reliabilities and 
significant correlations with other measures. 

The final point is suggested by considering the higher corre- 
lations between cloze and other tests of ESL proficiency when ac- 
ceptable rather than exact word scoring is used (Oiler, 1972? 
Hinofotis, 1980). Acceptable word cloze appears to yield the more 
reliable language proficiency measure. However, for reading com- 
prehension correspondences with standardized reading measures, ex- 
act word scoring is supported by previous research. These studies 
have revealed that e^act word cloze scores (with an every-fifth- 
word deletion scheme) of 57 to 6Q percent p.nd 38 to 44 percent, 
respectively, correspond to the 90 percent and 7 5 percent (inde- 
pendent and instructional) reading levels, respectively, on stan- 
dardized reading measures (Bormuth, 1968; Petersen et al. , 
1973). For the present, then, a practical recommendation is to 
obtain reading comprehension information with exact word scores 
using the fifth-word deletion scheme and to obtain language pro- 
ficiency information with the acceptable word scoring that seems 
intuitively appealing to teachers using an every-seventh-word de- 
letion scheme. This suggestion acknowledges that language profi- 
ciency and reading ability share a great deal of variance as 
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other research has indicated (Oiler and Perkins, 1978? Streiff, 
1978, 1979ai Oiler and Hinofotis, 1980 i Oiler and Streiff, in 
press) but are not identical constructs. One can certainly ex- 
hibit language proficiency without having reading ability but not 
the reverse. Two additional recommendations are pertinent. One 
is to use oral cloze as a direct measure of oral proficiency and 
written cloze for reading comprehension- Another suggestion on 
behalf of non-native speakers, for whom teachers prefer more le- 
nient deletion schemes, is further research into standardized 
reading *-est correspondences with more lenient cloze deletion 
schemes. 

A problem remaining for written cloze language proficiency 
scores, already tentatively solved for reading comprehension 
scores, is that of criterion levels of acceptable word cloze per- 
centage scores. The following section addresses this topic. 

TYPICAL NATIVE SPEAKER CLOZE SCORE LEVELS 

A question of interest is: Can one tell by non-native speak- 
ers' acceptable word scores how closely they perform to native 
speaker proficiency levels? Two alternatives have been suggested 
in previous work with cloze, and a third possibility arises out of 
the work of the seminar students, along with a review of previous 
studies . 

Approach One 

One alternative is proposed by Oiler et al. (1973). 
Testing Thai, Vietnamese, and English-speaking natives in both 
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their native languages and the first two groups, then, in their 
second language, English, they found that careful interpretation 
of a cloze test from one language to the ot* er yielded two cloze 
tests of comparable difficulty. By admistering a native language 
cloze test to students, one could get an idea of how well students 
could potentially perform in the second language. There are some 
practical problems with this suggestion. An ESL teacher may not 
be able to prepare a native language version of the test. Even 
with native speaker assistance, the problem isn't solved, since 
native speakers are not automatically equipped simply by virtue of 
being native speakers to produce careful interpretations so that 
both tests are of comparable difficulty. There are, of course, 
other situations in which this procedure may be a ready solution 
to problems of determining how well learners should ideally be ex- 
pected to perform in a second language. University foreign lan- 
guage classrooms are one such example. 

Approach Two 

Another route to having a standard for comparison for non- 
native speaker performance and progress toward a reasonable close 
test gcal is that employed by Streiff (1978). This approach in- 
volves testing a group of native speakers of the same age and/or 
grade background as that of a group of non-native speakers with 
whom comparison is desirable. In the case mentioned here, school 
officials of the non-native-speaking group requested cloze scores 
of a "run-of-the-mill middle class Anglo" group as a basis for 
comparison with their own students. This approach may have more 
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problems associated with it than the solution suggested before. 
Great care is demanded in material selection so that topics will 
be familiar or unfamiliar on an even basis? locating and obtaining 
permission to test a criterion group is also riddled with prob- 
lems. This approach hardly seems practical for classroom teach- 
ers, except in instances in which the second- or foreign-language 
learners attend institutions that also accommodate native speakers 
of the language. 

An Alternative 

There may be another route to answer the question of how well 
one's second- or foreign-language students compare with native 
speaker performance on cloze measures. A review of average native 
speaker scores was conducted among the seminar projects and also 
in several published studies of native speaker cloze testing. The 
average acceptable word scores of native speakers in both groups, 
seminar projects and published studies, reveal some regularities 
from the youngest test groups through the oldest. A total of 23 
cloze tests, administered from kindergarten through college lev- 
els, is represented in the general pattern represented in Figure 
3. Figure 4 presents the particular average score ranges found 
for each major grade level. In several instances, the published 
studies did not originally present average percentage scores. 
These percentage figures were derived from the number of cloze 
items and raw cloze scores reported in the studies (Oiler et 
al., 1973; Oiler and Conrad, 1971, • Stump, 1978; Streiff, 1978). 
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Figure 3 

GENERAL PATTERN OF NATIVE SPEAKER AVERAGE ACCEPTABLE 

WORD CLOZE SCORES 



Oral Cloze Written Clo2e 
Grades K-2 50%-70% 30%-50% 
Grades 3-6 70%-90% 50%-70% 

High School, College 70%-90% 

Figure 4 

AVERAGE SCORE SUMMARIES OF STUDIES REVIEWED 

Student Levels Average Cloze Score Range 

Oral Cloze Written Cloze 

Grades K-2 

7th-10th word deletion 54%-73% 34%-54% 

Grades 3-6 

7th-10th word deletion 72%-89% 59%-72% 

High School, College 

6th-t0th word deletion 74%-92% 
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Acknowledging the limited data available and the varying na- 
0 ture of the tests represented, the patterns appear quite regular 

enough to merit further attention. On written cloze, average na- 
tive speaker acceptable word scores increase in approximately 20 
percentage point increments from primary grades through middle 
elementary grades to high school and adult levels. A similar pat- 
tern emerges with oral cloze, where even fewer speaker studies are 
now available. In addition, the oral cloze patterns appear to 
correspond in a regular way to written cloze patterns, for exam- 
ple, the primary grade average oral score range mirrors the writ- 
ten score range of students of the middle elementary level. 

The general patterns on written cloze, 30 to 50 percent for 
primary grade students, 50 to 70 percent for middle elementary 
students, and 70 to 90 percent for high school and adult students, 
reflecting acceptable word scoring, may provide criterion ranges 
with which non-native cloze performance can be compared. Both 
patterns, those for written and for oral cloze, are presented as 
hypotheses, meriting further investigation. Firmer knowledge of 
this sort could be invaluable to teachers and program 
administrators in many situations. The present findings emerge 
from review of a wide variety of studies with many types of 
content, varying difficulty levels, and several deletion schemes. 
Considering these facts, the regularities are striking. In 
general, the atoove results were achieved under the following 
conditions: 

1 . acceptable word scoring; 

2. deletion schemes of every sixth to every tenth word; 
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3. for high school and adult students., written material in 
the range of approximately fourth to eighth grade diffi- 
culty? 

4. for elementary students, written cloze at the grade level 
of difficulty corresponding to grades in which students 
were enrolled,* and 

5. for oral cloze , material constructed from child dis- 
course, with normal features of child language intact 
(unedited) • 

Table 4 presents the data particular to each native speaker study 
or project included in this review. Figure 5 summarizes the aver- 
age percentage scores of non-native English speakers in the test 
project groups, for general comparison. 

CONCLUSIONS 

First, more systematic studies are suggested with cloze; for 
example, studies that systematically take student motivation into 
account, reveal correspondences between standardized reading lev- 
els and more lenient deletion schemes, student familiarity with 
topic analyzed in relation to cloze, and reveal oral and written 
cloze performance among native speakers of different ages. In ad- 
dition, cloze offers possibilities for imaginative research topics 
related to different language functions in varying contexts. 

Next, several major gains were apparent in the seminar stu- 
dents' testing skills. In brief reports of what they had learned 
from their own and other students' projects, the students men- 
tioned the following most frequently? recognition of their own 
contributions to error variance (from the tests themselves, to the 
test situation, to the test-takers). Specifically, they recog- 
nized the value of clear and consistent test instructions in both 
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Table 4 



NATIVE SPEAKER CLOZE SCORES 



Teacher/Tester 



Word 
Deletion 



Lang, of 
Subj ./Test 



Test 
Content 



Test 
Difficulty 



n of 
Items 



Score 



Mean Scores 



Kindergarten 
Sabino 



every 8th 



English 



oral, fables 
"The Turnip" 



low, first. 
Fry 



39 



accept, 
exact 



54% 46% 



Grade 1 



Gtreiff (1978) every 10th English 



every 7th 



English 



oral, child 
narrative on 
family, pets, 
activities 

written, 
first grade 
story: "On 
Mother's Lap" 



first grade, 
Fry 



50 



25 



accept, 
exact 



accept, 
exact 



61% 31% 



34% 24% 



Grade 2 



Streiff (1978) every 10th English 
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every 7th 



English 



oral, child 
narrative on 
family, pets, 
school 

written, 
story: "Andy 
and the Wild 
Worm" 



second grade, 
Fry 



50 



50 



accept, 
exact 



accept, 
exact 



73% 47% 



54% 40% 
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Teacher/Tester 



Word 
Deletion 



Table 4 (continued) 



Lang, of 
Sub j. /Test 



Test 
Content 



Test 
Difficulty 



n of 
Items 



Score 
IK** 



Mean Scores 



Grade 3 



Ferns and Ball every 7th 



Streiff (1978) every lUth 



every 5th 



English 



English 



English 



written, nat- 
ural science 
article on 
beavers 

oral, child 
narrative on 
family, pets, 
school 

written, 
story: "Small 
Boy Chuku" 



third grade, 
Fry 



third-fourth, 
Dale-Chall 



50 



50 



50 



accept, 
exact 



accept, 
exact 



accept, 
exact 



64% 



•?2% 



36% 



44% 



63% 26% 



Grade 4 



Ferns and Ball every 7th English 



Streiff (1978) every 10th English 



written, nat- third grade, 50 

ural science; Fry 

beavers 

oral, child 50 

narrative on 
family, school, 
child's job 



accept, 
exact 



accept, 
exact 



72% 



82% 



44% 



45% 



36 
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Table 4 (continued) 



Wbrd 

Teacher/Tester Deletion 

every 5th 



Lang, of 
Subj ./Test 



Test 
Content 



Test 
Difficulty 



n of 
Items 



Score 
Type 



English written, third-fourth, 
story r "Small Dale-Chall 
Boy Chuku* 



50 



accept, 
exact 



Mean Scores 



53% 



26% 



Grade 5 



Streiff (1978) every 10th 



every 5th 



English 



English 



oral, child 
narrative on 
family, pets, 
activities 

written, 
story: natu- 
ral science, 
baby polar 
bear 



fifth-sixth, 
Dale-Chall 



50 



50 



accept, 
exact 



accept, 
exact 



89% 



66% 



48% 



44% 



Grade 6 



Streiff (1978) every 10th 



every 5th 



English oral, child 50 accept. 84% 

narrative on exact 

family, school, 

activities 

English written, natu- fifth-sixth 50 accept. 59% 
ral science: Dale-Chall exact 
baby polar bear 



52 r * 



41% 
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Table 4 (continued) 



Teacher/Tester 



Word 
Deletion 



Lang, of 
Subj ./Test 



Test 
Content 



High School 
Angert 



Daniel 



Oiler 
et al. 

(1973) 



Test 
Difficulty 



£. Of 

Items 



every 7th 



every 7th 



every 6th 



every 6th 



every 6th 



English 



English 



Thai 



Viet- 
namese 



English 



written, 
Reader's 
Digest: 
"Joe's Eye 
Tooth* 

Folk tale: 
"Till 

Eulenspiegel" 

written, 
Reader's 
Digest: 
type source 
on Thailand 

written, 
Reader's 
Digest: 
type source 
on Vietnam 

written, 

Reader's 

Digest: 

"Thomas 

Edison" 



sixth grade, 50 
Fry 



fourth grade, 
Dale-Chall 



50 



40 



40 



40 



Score 



Mean Scores 



accept. 74% 63% 
exact 



accept. 93% 72% 
exact 



accept. 70% 



accept. 83% 



accept. 92% 
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Table 4 (continued) 



Teacher/Tester 



Word 
Deletion 



College 

Oiler and 
Conrad (1971) 



Lang, of 
Subj ./Test 



every 7th 



English 



Streiff (1978) every 10th English 



Test 
Content 



lest 
Difficulty 



written, 
"mat is a 
College?" 



written, 
on flying 



college 



fourth- 
fifth 



n of 
Items 



50 



50 



Score 
Type 



exact 



accept, 
exact 



Mean Scores 



44% (fresh- 
men) 

65% (gradu- 
ates) 

82% 57% 
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Figure 5 



SUMMARY: BILINGUAL AND ESL LEARNERS' 
ENGLISH CLOZE SCORES 



Kindergarten 

Exact scoring 
Acceptable scoring 

Grade 1 



Exact scoring 
Acceptable scoring 



Grade 2 



Exact scoring 
Acceptable scoring 

Grades 3-5 

Exact scoring 
Acceptable scoring 

High School and Adult 
Exact scoring 
Acceptable scoring 



28% (oral) 



16-52% (written) 
24-80% (written) 



44% (written) 49% (oral) 
36« (written) 35% (oral) 



21-31% (written) 
42-58% (written) 



15- 28% (written) 

16- 36% (written) 
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spoken-arid written form, of adequately familiarizing students in 
advance with the test task, pre-planning reasonable time periods 
for test administration, and the need for clearly printed test 
copies and for reasonable test difficulty levels* They also 
pointed out the merit in using the test results as teaching mate- 
rial. In general, insightful analytical comments too numerous to 
mention were forthcoming, many reflecting awareness of the inter- 
relationships of test, testing context, and test takers with the 
reliability and validity of tests. 

Finally, some major benefits were apparent with respect to 
interpreting cloze results. Teachers in bilingual elementary pro- 
grams observed that cloze was a valuable tool for assessing their 
students* performance in two languages. Primary teachers as well 
as teachers of adults became aware of the abilities and weaknesses 
of their students. Teachers of foreign languages at the high 
school level said they thought cloze deserved more widespread use 
in foreign language classrooms. Teachers of adults as well as of 
younger students saw cloze as a valuable teaching and testing 
tool. A number of students expressed regret that they didn't have 
cloze results from earlier in the academic year with which to 
compare their current results, recognizing the value in pre- 
testing and posttesting, and in periodic progress measurement. 
One student's statement at the end of a project period represents 
the typical view. She wrote, after testing first grade children, 
"I am now in the process of preparing another cloze test, one for 
a teacher down the hall. I feel I have just encountered something 
I should have known about long ago." 

45 
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